Non-monotone Exponential Time (NEXT) Model for the Longitudinal Trend of a Continuous Outcome in Clinical Trials.

The dose-response curve has been studied extensively for decades. However, most of these methods ignore intermediate measurements of the response variable and only use the measurement at the endpoint. In early phase trials, it is crucial to utilize all available data due to the smaller sample size. Simulation studies have shown that the longitudinal dose-response surface model provides a more precise parameter estimation compared to the traditional dose response using only data from the primary time point. However, the current longitudinal models with parametric assumptions assume the treatment effect increases monotonically over time, which may be controversial to reality. We propose a parametric non-monotone exponential time (NEXT) model, an enhanced longitudinal dose-response model with greater flexibility, capable of accommodating non-monotonic treatment effects and making predictions for longer-term efficacy. In addition, the estimator for the time to maximum treatment effect and its asymptotic distribution have been derived from NEXT. Extensive simulation studies using known data-generating models and using real clinical data showed the NEXT model outperformed the existing monotone longitudinal models.

CITIES: Clinical trials with intercurrent events simulator.

Although clinical trials are often designed with randomization and well-controlled protocols, complications will inevitably arise in the presence of intercurrent events (ICEs) such as treatment discontinuation. These can lead to missing outcome data and possibly confounding causal inference when the missingness is a function of a latent stratification of patients defined by intermediate outcomes. The pharmaceutical industry has been focused on developing new methods that can yield pertinent causal inferences in trials with ICEs. However, it is difficult to compare the properties of different methods developed in this endeavor as real-life clinical trial data cannot be easily shared to provide benchmark data sets. Furthermore, different methods consider distinct assumptions for the underlying data-generating mechanisms, and simulation studies often are customized to specific situations or methods. We develop a novel, general simulation model and corresponding Shiny application in R for clinical trials with ICEs, aptly named the Clinical Trials with Intercurrent Events Simulator (CITIES). It is formulated under the Rubin Causal Model where the considered treatment effects account for ICEs in clinical trials with repeated measures. CITIES facilitates the effective generation of data that resemble real-life clinical trials with respect to their reported summary statistics, without requiring the use of the original trial data. We illustrate the utility of CITIES via two case studies involving real-life clinical trials that demonstrate how CITIES provides a comprehensive tool for practitioners in the pharmaceutical industry to compare methods for the analysis of clinical trials with ICEs on identical, benchmark settings that resemble real-life trials.

Analysis of a Binary Outcome Dichotomized from an Underlying Continuous Variable in Clinical Trials.

Binary-valued outcome is often seen in many clinical trials across therapeutic areas. It is not uncommon that such binary endpoints are derived from a continuous variable. For example, in diabetes clinical trials, the proportion of patients with HbA1c< 7% is often investigated as one of the key objectives, where HbA1c is a continuous-valued variable reflecting the averaged blood glucose value from the previous three months. Most of the time, if not all, the mean of those binary endpoints were estimated directly through the binary variable defined by the corresponding cutoff. Alternatively, by the nature of the derivation, that quantity could also be estimated by leveraging the density of the underlying continuous variable and computing the area under the density curve up to a threshold. This paper provides a few methods in relation to density estimation. Extensive simulation studies were conducted based on real clinical trial data to compare these estimation approaches against the direct estimation of the proportions. Simulation results showed that the density estimation approaches in general benefited from a smaller mean squared error in early phase studies where the sample size is limited. The density estimation approach is certainly expected to introduce bias, however, a favorable bias-variance trade-off may make these approaches attractive in early phase studies.

Accurate Collection of Reasons for Treatment Discontinuation to Better Define Estimands in Clinical Trials.

BACKGROUND: Reasons for treatment discontinuation are important not only to understand the benefit and risk profile of experimental treatments, but also to help choose appropriate strategies to handle intercurrent events in defining estimands. The current case report form (CRF) commonly in use mixes the underlying reasons for treatment discontinuation and who makes the decision for treatment discontinuation, often resulting in an inaccurate collection of reasons for treatment discontinuation. METHODS AND RESULTS: We systematically reviewed and analyzed treatment discontinuation data from nine phase 2 and phase 3 studies for insulin peglispro. A total of 857 participants with treatment discontinuation were included in the analysis. Our review suggested that, due to the vague multiple-choice options for treatment discontinuation present in the CRF, different reasons were sometimes recorded for the same underlying reason for treatment discontinuation. Based on our review and analysis, we suggest an intermediate solution and a more systematic way to improve the current CRF for treatment discontinuations. CONCLUSION: This research provides insight and directions on how to optimize the CRF for recording treatment discontinuation. Further work needs to be done to build the learning into Clinical Data Interchange Standards Consortium standards. CLINICAL TRIALS: Clinicaltrials.gov numbers: NCT01027871 (Phase 2 for type 2 diabetes), NCT01049412 (Phase 2 for type 1 diabetes), NCT01481779 (IMAGINE 1 Study), NCT01435616 (IMAGINE 2 Study), NCT01454284 (IMAGINE 3 Study), NCT01468987 (IMAGINE 4 Study), NCT01582451 (IMAGINE 5 Study), NCT01790438 (IMAGINE 6 Study), NCT01792284 (IMAGINE 7 Study).

Using principal stratification in analysis of clinical trials.

The ICH E9(R1) addendum (2019) proposed principal stratification (PS) as one of five strategies for dealing with intercurrent events. Therefore, understanding the strengths, limitations, and assumptions of PS is important for the broad community of clinical trialists. Many approaches have been developed under the general framework of PS in different areas of research, including experimental and observational studies. These diverse applications have utilized a diverse set of tools and assumptions. Thus, need exists to present these approaches in a unifying manner. The goal of this tutorial is threefold. First, we provide a coherent and unifying description of PS. Second, we emphasize that estimation of effects within PS relies on strong assumptions and we thoroughly examine the consequences of these assumptions to understand in which situations certain assumptions are reasonable. Finally, we provide an overview of a variety of key methods for PS analysis and use a real clinical trial example to illustrate them. Examples of code for implementation of some of these approaches are given in Supplemental Materials.

Impact of Using A Mixed Data Collection Modality on Statistical Inferences in Decentralized Clinical Trials.

BACKGROUND: Decentralized clinical trials offer the promise of reduced patient burden, faster and more diverse recruitment, and have received regulatory support during the COVID-19 pandemic. However, lack of data accuracy or data validation poses a challenge for fully decentralized trials. A mixed data collection modality where onsite measurements are collected at key time points and decentralized measurements are taken at intermediate time points is attractive operationally. To date, the impact of decentralized measurements (which could presumably be less accurate) taken at intermediate time points on statistical inference on the primary or other key time points has not been evaluated. METHODS: In this article we evaluate the estimation and statistical inference for three scenarios: (1) all onsite measurements, (2) a mixture of onsite and decentralized measurements, and (3) all decentralized measurements, in the setting of a chronic weight management trial. We consider scenarios where decentralized measurements have additional within- and between-subject variabilities and/or bias. RESULTS: In the mixed modality setting, simulation studies showed that the estimation and inference for the key time points with onsite measurements have good properties and are not impacted by the additional variability and bias from intermediate decentralized measurements. However, estimates for intermediate decentralized time points for the mixed modality and estimates for the all decentralized modality measurements have increased variability and bias. CONCLUSION: Mixed modality trials can help achieve the benefits of decentralized clinical trials by reducing the number of onsite visits with little impact on statistical inferences for various estimands, compared to traditional (all onsite) clinical trials.

Analysis of an incomplete binary outcome dichotomized from an underlying continuous variable in clinical trials.

In many clinical trials, outcomes of interest are binary-valued. It is not uncommon that a binary-valued outcome is dichotomized from a continuous outcome at a threshold of clinical interest. To analyze such data, common approaches include (a) fitting a generalized linear mixed model (GLMM) to the dichotomized longitudinal binary outcome; and (b) the multiple imputation (MI) based method: imputing missing values in the continuous outcome, dichotomizing it into a binary outcome, and then fitting a generalized linear model to the "complete" data. We conducted comprehensive simulation studies to compare the performance of the GLMM versus the MI-based method for estimating the risk difference and the logarithm of odds ratio between two treatment arms at the end of study. In those simulation studies, we considered a range of multivariate distribution options for the continuous outcome (including a multivariate normal distribution, a multivariate t-distribution, a multivariate log-normal distribution, and the empirical distribution from a real clinical trial data) to evaluate the robustness of the estimators to various data-generating models. Simulation results demonstrate that both methods work well under those considered distribution options, but the MI-based method is more efficient with smaller mean squared errors compared to the GLMM. We further applied both the GLMM and the MI-based method to 29 phase 3 diabetes clinical trials, and found that the MI-based method generally led to smaller variance estimates compared to the GLMM.

Return-to-baseline multiple imputation for missing values in clinical trials.

Return-to-baseline is an important method to impute missing values or unobserved potential outcomes when certain hypothetical strategies are used to handle intercurrent events in clinical trials. Current return-to-baseline approaches seen in literature and in practice inflate the variability of the "complete" dataset after imputation and lead to biased mean estimators when the probability of missingness depends on the observed baseline and/or postbaseline intermediate outcomes. In this article, we first provide a set of criteria a return-to-baseline imputation method should satisfy. Under this framework, we propose a novel return-to-baseline imputation method. Simulations show the completed data after the new imputation approach have the proper distribution, and the estimators based on the new imputation method outperform the traditional method in terms of both bias and variance, when missingness depends on the observed values. The new method can be implemented easily with the existing multiple imputation procedures in commonly used statistical packages.

Estimating the treatment effect for adherers using multiple imputation.

Randomized controlled trials are considered the gold standard to evaluate the treatment effect (estimand) for efficacy and safety. According to the recent International Council on Harmonization (ICH)-E9 addendum (R1), intercurrent events (ICEs) need to be considered when defining an estimand, and principal stratum is one of the five strategies to handle ICEs. Qu et al. (2020, Statistics in Biopharmaceutical Research 12:1-18) proposed estimators for the adherer average causal effect (AdACE) for estimating the treatment difference for those who adhere to one or both treatments based on the causal-inference framework, and demonstrated the consistency of those estimators; however, this method requires complex custom programming related to high-dimensional numeric integrations. In this article, we implemented the AdACE estimators using multiple imputation (MI) and constructed confidence intervals (CIs) through bootstrapping. A simulation study showed that the MI-based estimators provided consistent estimators with the nominal coverage probabilities of CIs for the treatment difference for the adherent populations of interest. As an illustrative example, the new method was applied to data from a real clinical trial comparing two types of basal insulin for patients with type 1 diabetes.

A unified approach for evaluating the prediction of treatment effect across different types of endpoints.

Phase 2 and 3 development failure is one of the key factors for high drug development cost. Robust prediction of a candidate drug's efficacy and safety profile could potentially improve the success rate of the drug development. Therefore, systematic evaluation of the prediction is important for learning and continuous improvement of the prediction. In this article, we proposed a set of unified criteria that allow to evaluate the predictions across different endpoints, indications and development stages: standardized bias (SB), standardized mean squared errors (SMSE), and credibility of prediction. We applied the SB and SMSE to the predicted treatment effects for 54 comparisons in 5 compounds in immunology and diabetes.

Implementation of ICH E9 (R1): A Few Points Learned During the COVID-19 Pandemic.

The current COVID-19 pandemic poses numerous challenges for ongoing clinical trials and provides a stress-testing environment for the existing principles and practice of estimands in clinical trials. The pandemic may increase the rate of intercurrent events (ICEs) and missing values, spurring a great deal of discussion on amending protocols and statistical analysis plans to address these issues. In this article, we revisit recent research on estimands and handling of missing values, especially the ICH E9 (R1) Addendum on Estimands and Sensitivity Analysis in Clinical Trials. Based on an in-depth discussion of the strategies for handling ICEs using a causal inference framework, we suggest some improvements in applying the estimand and estimation framework in ICH E9 (R1). Specifically, we discuss a mix of strategies allowing us to handle ICEs differentially based on reasons for ICEs. We also suggest ICEs should be handled primarily by hypothetical strategies and provide examples of different hypothetical strategies for different types of ICEs as well as a road map for estimation and sensitivity analyses. We conclude that the proposed framework helps streamline translating clinical objectives into targets of statistical inference and automatically resolves many issues with defining estimands and choosing estimation procedures arising from events such as the pandemic.

Implementation of tripartite estimands using adherence causal estimators under the causal inference framework.

Intercurrent events (ICEs) and missing values are inevitable in clinical trials of any size and duration, making it difficult to assess the treatment effect for all patients in randomized clinical trials. Defining the appropriate estimand that is relevant to the clinical research question is the first step in analyzing data. The tripartite estimands, which evaluate the treatment differences in the proportion of patients with ICEs due to adverse events, the proportion of patients with ICEs due to lack of efficacy, and the primary efficacy outcome for those who can adhere to study treatment under the causal inference framework, are of interest to many stakeholders in understanding the totality of treatment effects. In this manuscript, we discuss the details of how to estimate tripartite estimands based on a causal inference framework and how to interpret tripartite estimates through a phase 3 clinical study evaluating a basal insulin treatment for patients with type 1 diabetes.

Defining estimands using a mix of strategies to handle intercurrent events in clinical trials.

Randomized controlled trials (RCTs) are the gold standard for evaluation of the efficacy and safety of investigational interventions. If every patient in an RCT were to adhere to the randomized treatment, one could simply analyze the complete data to infer the treatment effect. However, intercurrent events (ICEs) including the use of concomitant medication for unsatisfactory efficacy, treatment discontinuation due to adverse events, or lack of efficacy may lead to interventions that deviate from the original treatment assignment. Therefore, defining the appropriate estimand (the appropriate parameter to be estimated) based on the primary objective of the study is critical prior to determining the statistical analysis method and analyzing the data. The International Council for Harmonisation (ICH) E9 (R1), adopted on November 20, 2019, provided five strategies to define the estimand: treatment policy, hypothetical, composite variable, while on treatment, and principal stratum. In this article, we propose an estimand using a mix of strategies in handling ICEs. This estimand is an average of the "null" treatment difference for those with ICEs potentially related to safety and the treatment difference for the other patients if they would complete the assigned treatments. Two examples from clinical trials evaluating antidiabetes treatments are provided to illustrate the estimation of this proposed estimand and to compare it with the estimates for estimands using hypothetical and treatment policy strategies in handling ICEs.

Analysis of recurrent hypoglycemic events.

Hypoglycemia is a major safety concern for diabetic patients. Hypoglycemic events can be modeled based on time to recurrent events or count data. In this article, we evaluated a gamma frailty model with variance estimated by the inverse of observed Fisher information matrix, a gamma frailty model with the sandwich variance estimator, and a piecewise negative binomial regression model. Simulations showed that the sandwich variance estimator performed better when the frailty model is mis-specified, and the piecewise negative binomial regression sometimes fails to converge. All three methods were applied to a dataset from a clinical trial evaluating insulin treatments.

Proportion of treatment effect mediated by surrogate endpoints.

One of the central aims in randomized clinical trials is to find well-validated surrogate endpoints to reduce the sample size and/or duration of trials. Clinical researchers and practitioners have proposed various surrogacy measures for assessing candidate surrogate endpoints. However, most existing surrogacy measures have the following shortcomings: (i) they often fall outside the range [0,1], (ii) they are imprecisely estimated, and (iii) they ignore the interaction associations between a treatment and candidate surrogate endpoints in the evaluation of the surrogacy level. To overcome these difficulties, we propose a new surrogacy measure, the proportion of treatment effect mediated by candidate surrogate endpoints (PMS), based on the decomposition of the treatment effect into direct, indirect, and interaction associations mediated by candidate surrogate endpoints. In addition, we validate the advantages of PMS through Monte Carlo simulations and the application to empirical data from ORIENT (the Olmesartan Reducing Incidence of Endstage Renal Disease in Diabetic Nephropathy Trial).

Understanding and adjusting for the selection bias from a proof-of-concept study to a more confirmatory study.

It has long been noticed that the efficacy observed in small early phase studies is generally better than that observed in later larger studies. Historically, the inflation of the efficacy results from early proof-of-concept studies is either ignored, or adjusted empirically using a frequentist or Bayesian approach. In this article, we systematically explained the underlying reason for the inflation of efficacy results in small early phase studies from the perspectives of measurement error models and selection bias. A systematic method was built to adjust the early phase study results from both frequentist and Bayesian perspectives. A hierarchical model was proposed to estimate the distribution of the efficacy for a portfolio of compounds, which can serve as the prior distribution for the Bayesian approach. We showed through theory that the systematic adjustment provides an unbiased estimator for the true mean efficacy for a portfolio of compounds. The adjustment was applied to paired data for the efficacy in early small and later larger studies for a set of compounds in diabetes and immunology. After the adjustment, the bias in the early phase small studies seems to be diminished.

Estimation of group means in generalized linear mixed models.

In this study, we investigate the concept of the mean response for a treatment group mean as well as its estimation and prediction for generalized linear models with a subject-wise random effect. Generalized linear models are commonly used to analyze categorical data. The model-based mean for a treatment group usually estimates the response at the mean covariate. However, the mean response for the treatment group for studied population is at least equally important in the context of clinical trials. New methods were proposed to estimate such a mean response in generalized linear models; however, this has only been done when there are no random effects in the model. We suggest that, in a generalized linear mixed model (GLMM), there are at least two possible definitions of a treatment group mean response that can serve as estimation/prediction targets. The estimation of these treatment group means is important for healthcare professionals to be able to understand the absolute benefit vs risk. For both of these treatment group means, we propose a new set of methods that suggests how to estimate/predict both of them in a GLMMs with a univariate subject-wise random effect. Our methods also suggest an easy way of constructing corresponding confidence and prediction intervals for both possible treatment group means. Simulations show that proposed confidence and prediction intervals provide correct empirical coverage probability under most circumstances. Proposed methods have also been applied to analyze hypoglycemia data from diabetes clinical trials.

Estimation of group means using Bayesian generalized linear mixed models.

Generalized linear mixed models (GLMM) are commonly used to model the treatment effect over time while controlling for important clinical covariates. Standard software procedures often provide estimates of the outcome based on the mean of the covariates; however, these estimates will be biased for the true group means in the GLMM. Implementing GLMM in the frequentist framework can lead to issues of convergence. A simulation study demonstrating the use of fully Bayesian GLMM for providing unbiased estimates of group means is shown. These models are very straightforward to implement and can be used for a broad variety of outcomes (eg, binary, categorical, and count data) that arise in clinical trials. We demonstrate the proposed method on a data set from a clinical trial in diabetes.

Modeling the impact of preplanned dose titration on delayed response.

Dose titration becomes more and more common in improving drug tolerability as well as determining individualized treatment doses, thereby maximizing the benefit to patients. Dose titration starting from a lower dose and gradually increasing to a higher dose enables improved tolerability in patients as the human body may gradually adapt to adverse gastrointestinal effects. Current statistical analyses mostly focus on the outcome at the end-of-study follow-up without considering the longitudinal impact of dose titration on the outcome. Better understanding of the dynamic effect of dose titration over time is important in early-phase clinical development as it could allow to model the longitudinal trend and predict the longer term outcome more accurately. We propose a parametric model with two empirical methods of modeling the error terms for a continuous outcome with dose titrations. Simulations show that both approaches of modeling the error terms work well. We applied this method to analyze data from a few clinical studies and achieved satisfactory results.